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O \ Abstract 

j^ ' We consider a team game reward, and we derive a stochastic Pontryagin's maximum princi- 

ple for distributed stochastic differential systems with decentralized noisy information structures. Our 
methodology utilizes the semi martingale representation theorem, variational methods, and backward 
£> ■ stochastic differential equations. Furthermore, we derive necessary and sufficient optimality conditions 

■^j- ■ that characterize team and person-by-person optimality of decentralized strategies. 

-y*. ' Finally, we apply the stochastic maximum principle to several examples from the application areas 

t^J" \ of communications, filtering and control. 
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Index Terms. Team Games, Distributed Systems, Decentralized, Optimality Conditions, Maxi- 
mum Principle. 

I. Introduction 

We derive necessary and sufficient team game optimality conditions for distributed stochastic 
differential systems with decentralized noisy information structures. For noiseless information 
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structures, analogous optimality conditions are derived recently in [Q]| utilizing the representation 
of Hilbert space semi martingales and the stochastic Pontryagin's maximum principle of partially 
observed stochastic differential systems developed in [|2). However, the results obtained in Q] 
for decentralized noiseless information structures are not necessarily applicable to decentralized 
noisy information structures. In fact, there are certain technicalities that must be addressed 
when dealing with noisy information structures, which are inherited from the centralized fully 
observable versus partially observable stochastic optimal control fl3l-[fT2|. The main underlying 
assumption for centralized information structures, is that the acquisition of the information is 
centralized or the information acquired at different locations is communicated to each decision 
maker or control. 

When the system model consist of multiple decision makers, and the acquisition of information 
and its processing is decentralized or shared among several locations, then the different decision 
makers actions are based on different information [fl"3l . We call the information available for such 
decisions, "decentralized information structures or patterns" [fl4], [fT5l . When the system model 
is dynamic, consisting of an interconnection of at least two subsystems, and the decisions are 
based on decentralized information structures, we call the overall system a "distributed system 
with decentralized information structures". 

Over the years several specific forms of decentralized information structures are analyzed 
mostly in discrete-time [fT3l - ll25l . and more recently [|26l - ll34ll . However, at this stage the only 
systematic framework addressing optimality conditions for distributed systems with decentralized 
information structures is the one reported in [1] for decentralized noiseless information structures. 

In this paper, we consider a team game reward E2, ll25l . Il35l - ll37l . and we derive necessary 
and sufficient optimality conditions for distributed stochastic differential systems with decentral- 
ized noisy information structures. Our methodology utilizes the semi martingale representation 
theorem, variational methods, and generalizes the concepts utilized in HI, to derive optimality 
conditions for nonlinear stochastic distributed systems with decentralized noiseless information 
structures. From the practical point of view, the results of this part give optimality conditions 
in terms of forward and backward stochastic differential equations, and a Hamiltonian, called 
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"Hamiltonian System of Equations", which we use to compute the optimal decentralized decision 
strategies of several examples from the application areas of communications and control. 

The specific objectives of this paper are the following. 

(a) Derive team games Pontryagin's stochastic minimum principle (necessary conditions of 
optimality) for distributed stochastic systems with decentralized noisy information structures; 

(b) Introduce assumptions so that the necessary conditions of optimality in (a) are also 
sufficient, and relate the optimality conditions to person-by-person optimality conditions; 

(c) Apply the stochastic minimum principle to several examples from the application areas of 
communication and control. 

The rest of the paper is organized as follows. In Section |TT] we we formulate the distributed 
stochastic system with decentralized information structures. In Section UTH we introduce the 
variational equation and discuss its application in decentralized filtering and control. Section [V] 
is devoted to the development of stochastic optimality conditions for team games with decen- 
tralized information structures, consisting of necessary and sufficient conditions of optimality. In 
Section |V] we apply the minimum principle to various examples. The paper is concluded with 
some comments on possible extensions of our results. 

II. Distributed Stochastic Differential Team Games 

In this section we introduce the mathematical formulation of distributed stochastic differential 
systems, the noisy information structures available to the decision makers, and the definitions of 
collaborative decisions via team game optimality and person-by-person optimality. Although, the 
stochastic differential systems are driven by the Decision Makers (DMs) actions, our analysis 
includes unforced stochastic differential systems modeling distributed estimation. Therefeore, the 
term "decision maker" is used for distributed control as well as distributed estimation. 



The formulation presupposes a fixed probability space with filtration, ffi,F, {F 0) t : t G 
[0,T]},Pj satisfying the usual conditions, that is, (fi, F, P) is complete, F 00 contains all P- 
null sets in F. Throughout we assume that all filtrations are right continuous and complete |[38l . 

Define F T = {F , t : t E [0,T]}. 
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In our derivations we make extensive use of the following spaces. Let Lp T ([0, T], M. n ) C L 2 (fl x 
[0, T], cflP x dt, R n ) = L 2 ([0, T], L 2 {Q, W 1 )) denote the space of F T -adapted random processes 
{z(t) :tE [0,T]} such that 



E / {zfyl^ndt < oo, 

J[0,T\ 

which is a sub-Hilbert space of L 2 ([0,T], L 2 (ft,M n )). Similarly, let L| T ([0, T], £(M m ,M n )) C 
L 2 ([0,T], L 2 (f2,£(IR m ,IR n ))) denote the space of ¥ T — adapted n x m matrix valued random 
processes {£(£) : £ G [0,T]} such that 

E / |£(*)|£(Rm )B »)dt = E [ fr(E*(f)E(t))dt < oo. 

A. Distributed Stochastic System 

Next, we introduce the mathematical formulation of the stochastic system. On the fixed 



probability space f Q, F, {F ,t : t E [0, T]},P] we are given a distributed stochastic dynamical 
decision system. It consists of an interconnection of iV subsystems, and each subsystem i has, 
state space M n % DM action space A 1 C R d \ an exogenous state noise space W* = R m % an 
exogenous measurement noise space W = M fe % and initial state x l (0) = x l , defined by 

(51) x l (0) = x l : an W 1 * -valued Random Variable; 

(52) {W l (t) : t G [0, T]}: an M mi -valued standard Brownian motion which models the exogenous 
state noise, adapted to ¥ T , independent of x l (0); 

(53) {B l {t) : t E [0,T]}: an M fci -valued standard Brownian motion which models the exogenous 
measurement noise, adapted to F r , independent of {W l (t) : t E [0, T]}. 

The DM {u % : i E 1>n} take values in closed convex subsets of metric spaces {(M\ d) : i E 2^}. 
The decentralized partial information structure available to DM u % is generated by noisy obser- 
vation 

y\t)= [ h l (s,x\s),...x N (s),y\s),...y N (s))ds+ [ D^{s)dB\s), t E [0,T], ViEZ N , (1) 
Jo Jo 

where x % E W li is the state of subsystem % for i — 1, . . . , N. Notice that © models a channel 
with memory and feedback. Each subsystem is described by finite dimensional coupled stochastic 
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differential equations as follows. 



N 



dx\t) =f{t, x\t)y t )dt + <r% x'(t), ui)dw%t) + J2 f i3 ( f > x3 ( f )> u t) dt 

N 

+ J2 ° lj (t^ j (f)^t)dW 3 (t), x i (0) = x l , te(0,T], VieZ N . (2) 
i=i..7y« 
For decentralized communication and filtering applications the right side of © is independent 

of the DMs u\i = 1, ...,N. 



.,..., 



Since we considered a strong strong formulation, we define the filtration F^ = {F ,t : t G [0, T]} 
as follows. Introduce the a-algebras 



Wl t = a{(x\0),W\s),B l (s)):0<s<t}, Q$ = a{y i (s) : < a < *}, tG[0,T], i = 1, 
and the minimum a— algebras generated by these as follows 

N N 

t=l i=l 

Next, we introduce the admissible sets of decentralized decision strategies considered in this 
paper. 
(FIS): Feedback Information Structures. Let gf u = {Q y ^ : t G [0,T]} c {F 0i4 : t G [0,T]} 

denote the information available to DM i, Vi G Zjy- The admissible set of decentralized 

feedback strategies for DM i is defined by 

U^[0,T] = {«* G L^([0,T],R di ) : uj G A 4 C R di , a.e.t G [0,T], F-a.s.\, V* G Zjv, 

(3) 

where U^ u [0,T] is a closed convex subset of L| r ([0,T],R n ), for i = 1,2, . . . ,iV. 
Thus, an TV tuple of DM strategies is by definition 



.1 „.2 



U 



r )6U^"[0,T] = xf =1 ff""[0,T], 



and hence it is a family of TV functions, say, f [j,] (•),$(•), . . . ,fi^(-) ), t G [0,T], which 
are nonanticipative with respect to the information structures {Ql t : t G [0, T]},i = 
1,2,. ..,N. 

The information structure of each DM is decentralized, and may be generated by local or 
global subsystem observables. 
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(IIS): Innovations Information Structures. Let Q*' = a{P{t) : < t < T} denote the 
information available to DM i, \/i G Z N , where {P(t) : t E [0, T]} is the innovations of the 
process {y' l (t) : t e [0,T]} defined by 

I\t) = y\t)- J E{h\s,x\s),...,x N (s),y\s),...,y N (s))\g( s u }ds,te(0,T],VteZ N , 

(4) 
The admissible set of decentralized innovations strategies for DM i is defined by 

U /l '"[0,T] = lu l e U yl " u [0,T] : u\ is Q 1 ^ - adapted a.e.t E [0,T],¥ - a.s.V (5) 

An iV tuple of DM strategies is by definition (V, ..., w^) eUW"[0,T] = xf =1 V p ' u [0,T}. 
Define the augmented vectors by 

W = (W 1 , . . . , W N ) e M m , B = (B\ ...,B N )eR k ,u = (u\ . . . , u N ) e R d , x = (x\ ...,x N )e 

The distributed stochastic system dynamics are described in compact form by 

dx(t) = f(t,x(t),ut)dt + a(t,x(t),u t ) dW(t), x(0) = x , £g(0,T], (6) 

where / : [0, T] x R n x A^ — y R n denotes the drift and a : [0, T]xl n x A^ — ► C(W m , R n ) 

the diffusion coefficients. 

The distributed observation equations are described by the observation equation 

y{t)= [ h(s,x(s),y{s))ds+ [ D^s)dB(s), t E [0,T], (7) 

Jo Jo 

where h : [0, T] x W 1 x M. k — y R k is a function of the observations {y(t) :0<t<T}. 

B. Pay-off Functional and Team Games 

Consider the distributed system ©, © with decentralized partial information structures. Given 
auG IJ( N >> yU [0,T], define the reward or performance criterion by 



J(u) = J{u\u 2 ,...,u N ) =E<| J £{t,x{t),u t )dt + ip{x{T)\, (8) 

where £ : [0, T] x M n x U^ — > (—00,00] denotes the running cost function and </? : W 1 — > 
(—00, 00], the terminal cost function. Notice that the performance of the strategies is graded by 
a single pay-off functional. 
The distributed stochastic team optimization problem with N DM is defined below. 
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Problem 1. (Team Optimality) Given the pay-off functional ((#]), constraints ©, (0) the N tuple 



A 



of strategies u° = (u 1 ' , u 2 '°, . . . , u N '°) G lj( N )<y u [0, T] z's called team optimal if it satisfies 

Jiu 1 ' , u 2 >°, . . . , u N '°) < J{u\u\ ...,u N ), \/u=(u\u 2 ,...,u N )E U^*° [0, T] (9) 

Any w° G U^' y "[0, T] satisfying ([9]) w called an optimal decision strategy (or control) and the 
corresponding x°(-) = x(-; M°(-)),y°(-) = y(-;u°(-)) (satisfying ©, (0)) are called an optimal 
state process and observation process, respectively. 
Similarly, for u° = (u 1 ' , u 2 >°, . . . , w^ ) G U^' 1 " [0, T\. 

By definition, Problem [T|is a dynamic team problem with each DM having a different information 
structure (decentralized). An alternative approach to handle such problems with decentralized 
information structures is to restrict the definition of optimality to the so-called person-by-person 
equilibrium. 
Define 

j(v,« -i ) = j{u l ,u 2 ,...y- l ,vy +1 ,...,u N ), vggZjv. 

Problem 2. (Person-by-Person Optimality) Given the pay-off functional (©, constraints ©, ([7|) 
the N tuple of strategies u° = {u l, °,u 2 '°, . . . ,u N '°) G U^' yU [0,T] is called person-by-person 
optimal if it satisfies 

J(?/' , w-*'°) < J(«*, «-*•"), W G IP*'"[0,T], \/i G Z N . (10) 

Similarly for u° = (u 1 ' , u 2 '°, ..., /'") G OW' P [0, T]. 



The interpretaion of (1101 is that the variation of the z-th player is done while the rest of the 
players assume their optimal strategies. 

In the next remark, the previous team games formulation is discussed in the context of 
distributed estimation. 

Remark 1. In distributed estimation each subsystem is described by unforced coupled stochastic 
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differential equations 

N 

dx\t) =f{t, x^tydt + a\t, x l (t))dW\t) + Y, /*(*> x*{t))dt 

N 

+ J2 (y lj {t,x 3 {t))dw 3 {t), x i {o) = x i , te(o,T], \/ieZ N , (li) 

while the observations for each subsystem are described by ([7]). The distributed estimation objec- 
tive is to determine an N tuple of decision strategies u° = {u l, °,u 2 '°, . . . ,u N '°) E U^' y "[0, T] 
which is team optimal or person-by-person optimal (according to Problems [7J [2]) subject to 
constraints 4771) , (UJ. This distributed estimation problem formulated via team theory, is a 
generalization of the static team theory discussed in / l20l/ . / |221/ . / |2?1/ . / Tiol/ . However, we point 
out that for distributed filtering there is no reason to consider innovations information structures. 

III. Strong Solutions and Variational Equation 

In this section we introduce assumptions which will allow us to show existence of strong 
¥ T — adapted continuous solutions to © and ©. We also introduce the variational equation 
which is utilized to derive the stochastic minimum principle using the methodology in HI, 0. 
Let B™ ([0,T], L 2 (f2,M n )) denote the space of F T -adapted W 1 valued second order random 
processes endowed with the norm topology || ■ || defined by 

|| x || 2 = sup E|x(t)| 2 j„. 

te[o,T] 

The existence of strong solution is based on the following assumptions. 

Assumptions 1. (Main assumptions) The coefficients of the state and observation equations ©, 
are Borel measurable maps: 

/ : [0, T] x R n x A (iV) — ► R n , a : [0, T] x R n x A (7V) — ► £(M m , M n ), 

tf : [0, T]xR n xR k — ► R fc % Vi 6 Zjv- 

77?e,se satisfy the following basic conditions. 
There exists a K > such that 

(Al) |/(£, x, u) — f(t, z, u)\m.n < K\x — z\«n uniformly in u E A^ N \- 

(A2) \f(t, x, u) — f(t, x, i>)|ir« < K\u — v\ R d uniformly in x E W" 1 ; 
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(A3) 
(A4) 
(A5) 
(A6) 
(A7) 
(A8) 



f(t,x,u)\ R n < K{\ + |x| R » + |u| K d); 
a(t, x, u) - a(t, z, u) | £ ( R m iK n) < K\x - z\ 



a 



(t, x, u) — a(t, x, v)\j 



n) < K\U — V\ 



uniformly in u G A^ N '; 
uniformly in x G M. n ; 



o-(t,x,u)\c(R™,Rn) < K(l + \x\ R n + |u| R d); 
h(t,x,y)\ M k < K(l+ \x\ R n + \y\ Rk ); 
h(t,x,y) -h(t,z,y)\ Rk < K\\x-z\ R n + \y-y\ Rk J; 
(A9) For any a,b G C([0, T], R k *) the nonanticipative mapping u l = p} : [0, T] xC([0, T], R ki ) — ► 
A 1 satisfies the Lipschitz condition 

\n\t,a) - fx l (t,b)\ Rdl < K|a-&| c([0)T]jRfci) , i = 1,...,N. 

A10) D l ^(t) > 0,Vt G [0,T] and D*'3(.) w uniformly bounded, Vi G Zjv- 

The following lemma proves the existence of solutions and their continuous dependence on the 
decision variables. 

Lemma 1. Suppose Assumptions \B hold. Then for any Wo^-measurable initial state xq having 
finite second moment, and any u G U^ )W " [0, T], the following hold. 

(1) System ©, (0) has a unique solution (x,y) G S§°([0, T], L 2 (f2, IR n+fc )) having a 
continuous modification, that is, (x,y) G C([0,T],M n+fc ), P— as, Vz G Zjv- 

(2) 77ie solution of system ©, w continuously dependent on the control, in the sense 

that, as u^ — ► u l '° in W" u [0, T], Vz G Zyv, (x a ,Z/ a ) — ► (x°,y°) m Bj£([0,T], L 2 (fi,M n+fc )), 
Vz G Ztv- 
Similarly for u G UW' J "[0, T]. 

Proof: (1) Consider the augmented system X = (x, y) and the associated stochastic differ- 
ential equation of X. The proof for the first part of the lemma is classical and hence omitted. 
(2) Next, we consider the second part asserting the continuity of u to solution map u — y (x, y). 
Let {{u^ a : i = 1, 2, . . . , JV}, u°} be any pair of DM strategies from U^^" [0, T] x U^*" [0, T] 
and {x a , y a , x°, y°} denote the corresponding pair of solutions of the system ©, ©. Let u l,a — y 
^'°, % = 1, 2, . . . , N. We must show that (x a , y a ) — y (x°, y°) in B™ ([0, T], L 2 (fi, iT +fc )). By 
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the definition of solution to ©, it can be verified that 
x«(i) - x °(t) = J {/( S ,x a ( S ),<)-/( S ,x°( S ),<)}rf S 

+ 1 {a( S ,a: Q ( S ),<)-a( S ,x ( S ),<)}^( S ) + e?(t) + e^(t), t E [0,7], 

(12) 



where 



e?(*)=/ {/(s,x ( S ),<)-/( S ,x°( S ),<)]rf S (13) 

e a 2 (t)= j [a{s,x\s),u«)-a{s,x {s),u°S\dW{s). (14) 

Using the standard martingale inequality into (fl2T> . it follows from it and (Al), (A4) that there 
exist constants Ci,C 2 > such that 

E\x a (t)-x (t)gn <C X j K 2 E\x a (s) - x°{s)\ Rn + C 2 (E|e?(*)||„ + E|e£(i)|£»). (15) 

io 

Clearly, by the Cauchy-Schwartz inequality, and martingale inequality, it follows from (A2), 
(A5) that 

E|e?(£)|2„ <TE / |/(s,x ( S ),<)-/( S ,x°( S ),<)|2 n ^<TE / K 2 |<-<)|^fe, (16) 

70 JO 

E|e«(£)||„<4E f \a{s,x°{s),u«)-a( S ,x°(s),u°)\ 2 mm7Rn) ds<4E I K 2 \u a s - u% d ds. 
Jo Jo 

(17) 
Similarly, by (A8) 

E\y a (t)-y°(t)\ 2 Rh <Tj K 2 E(\x a (s) - x°(s)g n + \y a (s) - y°(s)&)tfc. (18) 

The integrands in the right side of inequalities ([TBI . (fi~7l converge to zero for almost all 
s E [0, T],P— a.s. Moreover, these integrands are dominated by integrable functions. Hence, 
by Lebesgue dominated convergence theorem the terms {e°, e 2 } converge to zero uniformly on 
[0,7]. Define p a {t) = E(\x a (s) - x°(s)\l n + \y a {s) - j/°(s)|£*Y Then by Gronwall inequality 
applied to p a , it can be shown that p a — > as u l,a — > u l, ° in HP*'" [0, T],\/i E TL^. The above 
analysis holds for innovations information structures. This completes the derivation. ■ 

Throughout the paper we assume existence of a minimizer u° E U^ N ^ yU [0, T] for Problem [Q 
For randomize (relaxed) strategies existence can be shown as in [0. 
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Next, we prepare to introduce the variational equation of the augmented system (x,y). 
Define the augmented vectors 

X=M6R"xR l , B^(B\B 2 ,...,B N )eR k , y = (y\y 2 , . . . ,y N ) e R k , 

and the augmented drift and diffusion coefficients, and terms in the pay-off associated with them 
by 



F(t,X,u] 



h(t,x,y) 



A 



f(t,x,u) 
h(t,x,y) 

h 1 (t,x,y) 



G(t,X,u) 



a 



a(t,x,u) 
£>*(*) 



h (t,x,y) 



D^^dia^ 1 -* (*),..., £>*'*(*)} 



L(t,X,u)=£(t,x,u), $(X) = <p(x). 



Then the augmented system is expressed in compact form by 

dW(t) 



dX{t) = F(t, X(t), u t )dt + G(t, X(t), u t ) 



dB(t) 



X(0)=X , te(0,T]. (19) 



For strategies U^' y "[0,T], since the state of the augmented system is X = (x,y), when 
considering variations of the state trajectory X, due to variation of u, there will be derivatives 
of w(-) with respect to y. To avoid this technicality we introduce the following assumptions. 

Assumptions 2. The diffusion coefficients a is restricted to the Borel measurable map a : 

[0,T] x W n x AW — > C(M. n ,W n ) (e.g., it is independent of u) and 
(All) cr(-, •) and cr~ 1 (-, ■) are bounded. 



Under the additional Assumptions |2] we can show the following Lemma. 

Lemma 2. Consider Problem [JJ under Assumptions [JJ [2] hold. Define the a— algebras 

^o1 W = *{(*«>), W{s),B{*)) :0<s<t}, Jff = 

Ifue V^' yU [0,T] then F^ W ' B = jtf»* W e [0,7]. 



a <(x(s),y(s)):0<s<t , We[0,T] 



Proof: This follows directly from Assumptions |2] and the invertibility of £)*(•), Vi G Z 



N- 
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Recall that {x(t), y(t) : £ G [0, T]} are the strong F T — adapted solutions of the state and observa- 
tion equations. Under the conditions of Lemma|2]for any u % G W % ' u [0, T] which is Q\, —adapted 
there exists a function </> l (-) measurable with respect to a sub-cr— algebra of T^ t C T^ t ' 
such that uj(w) = 4 (£,x(O), B(- A*, w), W(- A*, w)),P- o.s. cj e fi,Vt G [0,T],i = 1,...N. 
Define all such adapted nonanticipative functions by 

% a [0,T] = iu l G Z£ r ([0,T],R*) : wj G A* C M d % a.e.t G [0,T], P-a.s.}, Vi G Zjy. (20) 

Next, we introduce the following additional assumptions. 

Assumptions 3. W i,u [0,T] is dense in I? na [0,T], Vi G Z^. 

Under Assumptions [3] we can show the following theorem. 

Theorem 1. Consider Problem\l}under AssumptionsU\\3\ Further, assume I is Borel measurable, 
continuously differentiable with respect to (x, u), and <p is continously differentiate with respect 
to x, and there exist K\ , K 2 > such that 

\£ x (t,x,u)\-Rn + \e u (t,x,u)\ R d\ < Kt{l + \x\ru + \u\ R d), \ip x (T,x)\m < K 2 (l + |x| R »). 

Then 

inf J(u) = inf J(u). (21) 

«exf =1 UL[0,T] «exf =1 U^"[0,T] 

Proof: Since Assumptions [3] holds, it is sufficient to show that as u l,a — > u l in U na [0, T], 
Vz G Ztv, then J(w a ) — > J{u). From the derivation of Lemma[Q we can show that E sup sG r t i \x a (s) 
x(s)\ R n converges to zero as a — > 00, hence it is sufficient to show that \J(u a ) — J(u)\ also 
converges to zero, as a — > 00. By the assumptions on {£, (p}, and by the mean value theorem 
we have the following inequality. 

I J(u a ) - J(u)\ <K X e{ / (\x a (t)\ Rn + |<| Rd + |x(t)| R » + \u t \ Rd + l) 

L J[0,T] V J 

■ (\x a (t) -x(t)\ Rn + \u" -u t \ R d)dt\ 

+ K 2 E{ (\x a (T)\ Rn + \x(T)\ Rn + l) \x a (T) - x(t)\ Rn y (22) 

Since Esup se r 0)t i \x a (s) — x(s)\ R n — > as a — ► 00, then \J(u a ) — J(u)\ also converges to 
zero, as a — > 00. 
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The point to be made regarding Theorem [T] is that if u G U^' y "[0, T] achieves the infi- 
mum of J{u) then it is also optimal with respect to some measurable functionals of subsets 
{(x(0), (W(s),B(s)) : < s < T}. Consequently, the necessary conditions for u E U (7V) ^"[0, T] 
to be optimal are those for which u G x^ 1 U na [0, T] is optimal. 

Remark 2. Strategies adapted to the innovations process u % G Q^' u , Wi G Z, N are often utilized 
to derive the separation theorem of partially observed stochastic control problems. It is well 
known that {P' u (t) : t G [0, T}} is an Gt —adapted Wiener process. Moreover, if the innovations 
process and observation process generate the same a— algebra, Gq s = Go's", and the innovations 
process is independent of u, then P' u (t) = P'°(t),Wt G [0,T]. Define the a— algebra Got = 
a{P ,0 (s) : < s < t},t G [0, T]. Under these conditions the necessary conditions for u G 
U' JV ' ,y "[0, T] to be optimal are those for which u G U^' 7 [0,T], defined by 

UW' J °[0,T] = |u (JV) ' y " : w* w ^ t ° - adapted, Vi G Zjy}. (23) 

./Vote ?/?a? Assumptions El are nof required for distributed filtering applications because the 

. i 

decentralized information structures Gt are independent of u, and hence it is not very difficult 
to show 0^ = Gt i ^ e ^n- 

After deriving the necessary conditions we also show that under certain convexity conditions that 
these are also sufficient. Consequently, for the sufficient part we do not require Assumptions \3\ 

For the derivation of stochastic minimum principle of optimality we shall require stronger 
regularity conditions on the maps {/, a, h}, as well as, for the running and terminal pay-offs 
functions {I, (p}. These are given below. 

Assumptions 4. E|x(0)|r™ < oo and the maps of {f,a,£,(p} satisfy the following conditions. 
(Bl) The map f : [0, T] x M. n x A^) — ► M. n is continuous in (t,x,u) and continously 

differentiable with respect to (x, u); 
(B2) The map a : [0, r]xl n xA^ — > £(R m ; R n ) is continuous in (t, x, u) and continously 

differentiable with respect to (x, u); 
(B3) The map h : [0, T] x ~R n x M fc — y IR fc is continuous in (t, x, y) and continously 

differentiable with respect to (x,y); 
(B4) The first derivatives {f x , fu,^x,cr u } are bounded uniformly on [0,T]xK"x A^ N \- 
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n w Trpfe. 



(B5) The first derivative {h x ,h y } are bounded uniformly on [0,T] 

(B6) 77ie map* £ : [0,T] x M n x A^ — > (—00,00] is Borel measurable, continuously 
differentiate with respect to (x, u), the map 92 : [0, T] xR n — > (—00, 00] is continously 
differentiable with respect to x, and there exist K\ , K 2 > such that 

\£ x (t,x,u)\ R n + \£ u (t,x,u)\ R d\ < Ki(l + |x| R » + \u\ R d), \<p x (T,x)\ R n < K 2 (l + |x|r») 

(B7) Conditions (A9), (A10) of Assumptions [7] hold. 

Consider the Gateaux derivative of G with respect to the variable at the point (t, z, v) e [0, T] x 
Rn +k x A (N) in the di rect i on v G R n+k defined by 

Gx(t,z,v;rj) = lim -\ G(t, z + er],u) -G(t,z,v)\, t e [0,71 

Note that the map r\ — > Gx(t, z, v\ if) is linear, and it follows from Assumptions @] (B3), (B5) 
that there exists a finite positive number (5 > such that 

\Gx(t,z, u;r])\ c(Rm +k tR n+k- } < (3\rj\ R n+k, te [0,T]. 

In order to present the necessary conditions of optimality we need the so called variational 
equation. Suppose u° = (u 1 ' , u 2 '°, . . . , u N, °) E U^' 7 "[0,T] denotes the optimal decision and 
u = (u\u 2 , ...,u n )e U (7V) - /u [0,T] any other decision. Since U /I,u [0,T] is convex Vz 6 Z N , it 
is clear that for any e e [0,1], 



i,e A 
U,' = 



ui'° + £(ui-ui'°) eU Ji '"[o,T], \/ieZ N . 



Let X £ (-) = X £ (-;u £ (-)) and X°{-) = X°(-;u°(-)) e B™ t ([0, T], L 2 (fi,M n+fc )) denote the 
solutions of the system equation (fT9b corresponding to -u e (-) and w°(-), respectively. Consider 
the limit 

Z(t)=\im-\x £ (t)-X°(t)}, te[0,T\. 

We have the following result characterizing the the variational process {Z(t) : t 6 [0,T]}. 

Lemma 3. Suppose Assumptions \4\ hold. For strategies U^ ,7 "[0,T] the process {Z(t) : £ G 
[0, T]} w an element of the Banach space -B|^,([0, T], L 2 (f2, M n+fc )) and ?Y z's z/ze unique solution 
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of the variational stochastic differential equation 

dZ(t) = F x (t, X°(t),u°)Z(t)dt + G x (t, X°(t),u°; Z(t)) 



dW{t) 
dB{t) 



N 



N 



+ J2 FA*, X°(t), OK - ul'°)dt + J2 G u *(t, X°(t),u° t ; u\ - u\°) 



i=l 



i=l 



dW(t) 
dB(t) 



, Z(0) = 0. 
(24) 



having a continuous modification. 

Under the addition Assumptions \3\ the above statements hold for strategies U^' y "[0, T]. 
Moreover, < [24\) is the variational equation for distributed filtering applications (without imposing 
Assumptions El). 

Proof: This follows from Q] by considering the augmented system. 

■ 
Using the variation equation of Lemma |3l we note that the results given in HI for nonrandomized 
strategies are directly applicable to the augmented system (fl9l) . In fact one can also consider 
randomized strategies as in [[T). 

IV. Optimality Conditions for Noisy Information Structures 

In this section we derive necessary and sufficient optimality conditions for the team game of 
Problem \\\ In view of the results obtained in the previous section, specifically, Lemma [3J the 
stochastic minimum principle of optimality for Problem [TJ described in terms of the augmented 
system (fT9l ), follows directly from the results in [0Q. 



Before we introduce the optimality conditions we define the Hamiltonian system of equations. 
To this end, define the Hamiltonian 



M : [0, T] x R n x W 1 x £(R m , W 1 ) x A {N) 



by 



W{t, x, i;, q n , u) = (f(t, x, u), ij>) + tr(q* u a(t, x)) + £{t, x,u), tE [0, T}. (25) 

For any u G U^'» u [0, T], U^' 7 "[0, T], the adjoint process is 



ty,qii,<h2) E L'([0,T},R n ) x L 2 ¥ ([0,T},C(R m ,R n )) x L'([0, T], £(R\ R n )) 
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and satisfies the following backward stochastic differential equation 
dip(t) = - f*(t, x{t), ut)ip{t)dt - V qil (t)dt - £ x (t, x(t),ut)dt 
+ q u (t)dW(t) + q 12 (t)dB(t), te[0,T), 
= -U x {t,x{t),i;{t),qn{t),u t )dt + qn{t)dW{t) + q 12 {t)dB{t), te [0,T), (26) 
V(T) =<p x (x(T))> (27) 

where V qil G L^([0,T],R") is given by (V qil {t),() = tr(q* u (t)a x (t,x(t);C)),t G [0,T] (e.g., 
Vqn(t) = Ek=i (^(MW))*?!?^), * e [0,T], o-( fe ) is the kth column of a, a x k) is the 
derivative of a^ with respect to the state, q[ x is the kth column of ^n, for k — 1, 2, . . . , m). 
The state process satisfies the stochastic differential equation 

dx{t) = f(t,x(t),u t )dt + a(t,x(t))dW(t), te (0,T], 

= H v ,(t,x(t),^(t),gn(t),M t )rft + a(t,x(t))dW(t), te(0,T], (28) 

x{0) = x (29) 

The above Hamiltonian system of equations is expressed in terms of the original distributed 
system of equations ©, ©, and it is obtained by first deriving the Hamiltonian system of 
equations for the augmented system (fT9l (we shall clarify this step in the next section). 



A. Necessary Conditions of Optimality 

We now prepare to derive the necessary conditions for team optimality. Specifically, given 
that u° G U (7V) ' y "[0,T] or u° G U (Ar) ' 7 "[0, T] is team optimal the question we address is whether 
it satisfies the Hamiltonian system of equations (f25T)-(f29l). 
By utilizing [1J we have following necessary conditions. 

Theorem 2. (Necessary conditions for team optimality) Consider Problem [7] under Assump- 
tions El and A 1 a closed, bounded and convex subset of M ki , % — 1, . . . N. 

For an element u° e UW' J "[0,T] with the corresponding solution x° G £^([0,T],L 2 (fi,R n )) 
to be team optimal, it is necessary that the following hold. 

(1) There exists a square integrable semi martingale m° with the intensity process (if) , q° t , q\ 2 ) G 
L 2 ¥T ([0,T],M n ) x Lj T ([0,T],C(R m ,R n )) x Lj T ([0,T],C(R k ,R n )). 
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(2) The variational inequality is satisfied: 



N rT 



Ve/ (U ul (t,x o (t),r(t),q° ll (t),u o t ),ul-ul> o }dt>0, Vn6U«[0,T], (30) 

(3) r/ ? eproce55(^^^,g 1 o 2 )eL2 T ([0,r],M«)xL| T ([0,T],£(M m ,M"'))xL| T ([0,T],£(M fc ,R")) 
zs a unique solution of the backward stochastic differential equation rf2o1) . ft27\) such 
that u° £ U^' /u [0, T] satisfies the point wise almost sure inequalities with respect to 
the a-algebras Q 1 ^ C F 0)f , t £ [0, T],i = 1, 2, . . . , N : 



{E{m u ,(t,x (t),r(t),q u (t),u° t )\g^ t j,ul-u l t '°) >0, 

VV £ A\a.ei £ [0,T],PL 7! , u o - a .s.,z' = l,2,...iV. (31) 

Under the additional Assumptions \3\ the results also hold for strategies U^ ,y "[0,T] vvzY/z con- 
ditional expectation taken with respect to Q^ t . 

For distributed filtering strategies are u £ V^' y [0,T] with conditional expectation taken with 
respect to Q^' t . 

Proof: The derivation consists of two steps. The first step utilizes [1] to derive the optimality 
conditions for the augmented system (fl9l) . Hence, by direct application of [1] we have the 
following. 
Define the Hamiltonian of the augmented system ( |T9l) 

H : [0,T] x R n+k x R n+k x £(R" +fe ,M n+fe ) x A (A ° — ► R 

by 

H(t,X,V,M,u) = (F(t,X,u)X) + tr(M*G(t,X)) + L(t,X,u), t£[0,T]. (32) 

Then the result of [Q] for nonrandomized strategies apply to the system (fl9l) , hence for any 
u £ x^ 1 U y *' u [0,T] or w £ x^ =1 U / '' u [0,T] the adjoint process of the augmented system exists 
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and satisfies the following backward stochastic differential equation. 



d*(t) = -F x {t, X(t), u t )m{t)dt - V Q (t)dt - L x {t, X(t),u t )dt + Q(t) 



dW(t) 
dB(t) 



, te[0,T), 



-Hx(t,X(t),V(t),Q(t),u t )dt + Q(t) 



dW(t) 

dB(t) 



, *(T) = $ x (X{T)),tG[0,T), 

(33) 



where V Q is given by (V Q (t), C) = tr(Q*(t)G x (t,X(t); ()),te [0,T]. 

The second step translates the necessary conditions of the augmented system to the original 
system ©, ©. To this end, we introduce the following decompositions which will lead to a 
simplified Hamiltonian system of equations. 



* 



v 


, Q = 


?11 <?12 


_ c _ 




Q21 Q22 



(34) 



By utilizing this decomposition it can be shown that if) satisfies (1261) . (|27l) . The second component 
of \P in (l34l) satisfies the following equation 



d((t) = q 2 i(t)dW(t) + q 22 (t)dB(t), ({T) = 0, t G [0,T). 



(35) 



Since this equation has terminal condition C(T) = 0, and its right hand side martingale terms are 
orthogonal, then necessarily, 921 (t) = 0,q22(t) = 0,W G [0,T],a.s., which imply £(t) = 0,Vt G 
[0, T],a.s. Finally, statements (l)-(3) are obtained from equivalent statements of the augmented 
system JT). ■ 

It is interesting to note that the necessary conditions for a u° G U^ ,J/ "[0,T] or u° G 
U^' /u [0,T] to be a person-by-person optimal can be derived following the methodology of 
Theorem |2] and that these necessary conditions are the same as the necessary conditions for the 
team optimal strategy. These results are stated as a Corollary. 

Corollary 1. (Necessary conditions for person-by-person optimality) Consider Problem^under 
the assumptions of Theorem\2\ For an element u° G U^' 7 " [0, T] with the corresponding solution 
x° G B^ ([0, T], L 2 (Q,M n )) to be a person-by-person optimal strategy, it is necessary that the 
statements of Theorem [2] (1), (3) hold and statement (2) is replaced by 
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(2') The variational inequalities are satisfied: 

e[ (U u ,(t,x°(t),ij o (t),q^(t),u o t ),ui-ui' o )dt>0, W eW' Ii,u [0,T], VieZjv. 
Jo 

(36) 

Under the additional Assumptions \3\ the results also hold for strategies U^ ,y "[0,T] with con- 
ditional expectation taken with respect to Q$ t . 

For distributed filtering strategies are u E U^' y [0,T] with conditional expectation taken with 
respect to Ql ± . 

Proof: The derivation is based on the procedure of Theorem |2] which is completely described 
in El ■ 

The following remark helps identifying the martingale term in the adjoint process. 

Remark 3. According to £/]/, the Riesz representation theorem for Hilbert space martinagles, 

determine the maritingale term of the adjoint process M t = J Q $j(5)G(s,X"(s)) 

dB(s) 

dual to the first martingale term in the variational equation ft24\) . hence Q in the adjoint equation, 
is identified as Q(t) = ^x(t)G(t,X(t)). By translating this to the original system then q±i = 

i^xO~,qi2 = ipyDz, provided the derivatives ifj x ,ip y exist. 

B. Sufficient Conditions of Optimality 

In this section, we show that the necessary condition of optimality (I3TI) is also a sufficient 
condition for optimality, under a convexity conditions on the Hamiltonians and the terminal 
condition. 



Theorem 3. (Sufficient conditions for team optimality) Consider Problem [7] with strategies from 
U^' 7 "[0,T] (respectively V^' yU [0,T]), under Assumptions \4\ and A 1 a closed, bounded and 
convex subset ofM. ki , i = 1, . . . N. Let (x°(-),u°(-)) denote an admissible state and decision pair 
and let ifj°(-) the corresponding adjoint processes. 
Suppose the following conditions hold. 

(C4) H(t, ■,ifj,q 11 ,-),te [0, T] is convex in (x, u) e R n x A^; 

(C5) ip(-) is convex in x e M n . 
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Then (x°(-), u°(-)) is a team optimal pair if it satisfies 071) (respectively 071 ) with conditional 
expectation taken in terms of Q^ t , % — 1, . . . N). 

Proof: Let u° E U^' y "[0, T] denote a candidate for the optimal team decision and u E 
U( Ar )-^[0,T] any other decision. Then 

J{u°) - J(u) = l{ / (*(*, x°(t),<) - £(£, x(t), ut))cft + (^°(T)) - p(a;(T))) |. (37) 

By the convexity of </?(•) then 

^(x(T)) - ^(x°(T)) > (^(x°(T)),x(T) -x°(T)>. (38) 

Substituting ([38]) into (fJU) yields 

J(«°) - •/(«) < e{(^(x°(T)),x°(T) - x(T))} 

+e| / fex°(t),M°)-^,x(i),w t )^t|. (39) 

Applying the Ito differential rule to (ip°,x — x°) on the interval [0, T] and then taking expecation 
we obtain the following equation. 

E[(r(T),x(T) - x°(T))} =E{(r(0),x(0) - x°(0))} 

+e[J (-f:(t,x°(t),u° t )r(t)dt -v q0 jt) - e x (t,x°(t),u° t ),x(t) - x°(t))dt 

+ e{ J (ip°(t), f(t, x{t), ut) - f{t, x°(t), u°))dt 

+ E{ f tr(qt[°(t)a(t, x(t)) - q*A°(t)a(t, x°{t))dt 

= ~e[J {W x {t,x\t),r{t)Ali{t)X)At)-x°{t))dt 

+ E{ J (ip°(t), f(t, x(t), ut) - f(t, x°(t), u°))dt 

+ E{ J tr(qt[°(t)a(t, x(t)) - ql[°(t)a(t, x°(t)))dt \ (40) 



Note that tp°(T) = <p x {x°(T)). Substituting (g0j» into ([39]) we obtain 

J(u°) - J(u) <E{ J [HO, x°(t), r(t), q° n (t), u°) - U(t, x(t), r(t), q°n(t),u t )] dt} 

-e{J (U x (t,x°(t),r(t),q u (t),u° t ),x (t) - x(t))dt}. (41) 
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Since by hypothesis HI is convex in (x,u) £l"x A^, then 

H(t, x(t),r(t), &(*), u t ) - H(t, x°(t), iP°(t), q°n(t),u° t ) 

N 

>J2(^(t,x°(t),r(t),q 11 (t),u t ),u l -ul'°) 

i=l 

+ (U x (t,x°(t),i(j (t),Q (t),u t ),x(t) - x°(t)), te[0,T] (42) 

Substituting (g^> into gB yields 

N T 

J(u°) - J(u) < -E[J2 / (^(t,x (t),r(t),q u (t),u t ),ui-u^ )dty (43) 

By (I3TT) and by definition of conditional expectation we have 

E{/ A j(a;)(e ul (t,x (t),^ (t),g 1 1 (t),<),^- M r}} 



= E|/ 4 j(w)E|(H ui (t,a; (t),V o (t),^ 1 (t),« t o ),t4-wl' o >l^t )} > 0, V^ e ^ , Vz e Z„. 

(44) 

Hence, (H u i(i,a; (t), V°(*)i3ii(*)> «*)•«« ~ u t'°) > 0,Vuj e A*,a.e.t e [0,T],F - a.s.,i = 
1, 2, . . . , N. Substituting the this inequality into (l43l) gives 

j(u°)<j(u), VM6#^[o,r]. 

Hence, sufficiency of ( f3TT ) with conditional expectation taken in terms of Qq \ , i — 1, . . . N is 
shown. For U^ Ar - l ' / "[0,T] the derivation is identical. ■ 

Since the necessary conditions for team optimal and person-by-person optimal are equivalent 
(this follows from Theorem |2] Corollary [T), then one can go one step further to show that under 
the conditions of Theorem [3] that any person-by-person optimal strategy is also a team optimal 
strategy. 

We conclude our discussion on team and person-by-person game optimality conditions for 
distributed stochastic differential systems with decentralized noisy information structures, by 
stating once again that the results derived are also applicable to distributed estimation problems 
(see Remark [U) with strategies taken from U^' y "[0,T]. 
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V. Applications in Communication, Filtering and Control 

In this section we investigate various applications of the optimality conditions to communi- 
cation, filtering and control applications. For most applications we give explicit optimal team 
strategies, when the dynamics and the reward have the structures defined below. Throughout, we 
assume validity of the convexity conditions of Theorem |3] (C4), (C5), why imply sufficiency 
of (1311) with conditional expectation taken in terms of Qq t , i = 1 , . . . N, and strategies taken 
fromUW^[0,r]. 

Definition 1. (Team games with special structures) We define the following classes of team 
games. 

(NF): Nonlinear Form. The team game is said to have "nonlinear form" if 

N 

f(t,x,u)=b(t,x) + g(t,x)u, g(t,x)u = ^2g ( - j) (t,x)u\ (45) 

.7=1 

a(t,x) = [aM(t,x) aW(t,x)... a^ N \t,x)] (46) 

£(t, x, u) =-{u, R(t, x)u) + -A(t, x) + (u, v(t, x)), (47) 



where 



N N N 



(u,R(t,x)u) = ^2^2 ui '* ''Rij(t, x)u j ', (u,r](t,x)) = y~V'V(f,:r), 

4 = 1 j=l 8=1 

and a^ (•, •) is the ith column of an nxm matrix er(-, •), for i — 1, 2, . . . , m, R(-, •) is symmetric 
uniformly positive definite, and A(-, •) is uniformly positive semidefinite. 

(LQF): Linear-Quadratic Form. A team game is said to have "linear-quadratic form" if 

f(t,x,u)=A(t)x + B(t)u, a(t,x,u) = G(t), (48) 

£{t, x) =-(u, R(t)u) + -{x, H(t)x) + (x, F(t)) + (u, E(t)x) + (u, m(t)), (49) 

and R(-) is symmetric uniformly positive definite and H(-) is symmetric uniformly positive 
semidefinite. 

Below we compute the optimal strategies for the two cases of Definition [TJ First, we introduce 
the following definitions. 

vfi°(t) = E(ui'°\g( t u °\ v£°{t) = Vector{^°(t), . . . , vF*°(t)}, i,j = l...,N, 
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u°(t) = Vectoring), . . .,u N <°(t)}, x°{t) = Vector{E(x°{t)\g£f\ . . . ,E(x (t)\g$' u °)}, 



r(t) = Vector{EU°(t)\gi; ),..., EU°(tW , t } 



R®(t) 



A 



Ra(t), . . . , RiN(t) 



, E&{t) 



En(t), . . . ,E iN (t) 



z = l, ...,7V. 



Case NF. 
Utilizing the definition of Hamiltonian of Theorem |2l its derivative is given by 

H u (t,x,ip,q ll ,u) = g*(t,x)tp + R(t,x)u + r](t,x), (t, x) e [0,T] x M. n . (50) 

The explicit expression for u l t '° is given by 



TV 



U 



^E[R ti (t,x°(t))\gl t 



E(r,%x°(t))\g? 



o.t 



J2 E^txitM^Gi 



-E[g^*(t,x°{t))r{t)\Qtt )}, n g y T °-a.s., i = 1, 2, . . . , N. 



(51) 



Special Case. Suppose R(t,x) = R(t), e.g., independent of x. Since both sides of (ISTT) are 

o - o 

^o t —measurable taking conditional expectations of both side with respect to ^ t ~ gives the 

expression 

-i N 

ifi**(t) = - fat)} {E(ri%ar(t))\GC)+ E fy(*V A °(*) 



E( 5 «'*(t,x°(t))^(t)i^; 



J=^J¥=i 



»0,< 



a.s., i= 1,2, ...,/V. (52) 



The last equation can be written in the form of a fixed point matrix equation with random 
coefficients. We discuss this below. 



Case LQE 

For a team game is of normal form then from the previous optimal strategies one obtains 

TV TV 

-R^\t){ m \t) + j2Mm(^°(t)\gC) + y, RiAm(<\eC° 



i,o 



B^*(t)EU°(t)\g y 



Kt / I ' 



\qV> 
y 0.i 



a.s., i = 1, 2, . . . ,N. 



(53) 
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Similarly as above, (1531 ) can be put in the form of fixed point matrix equation as follows. 

diag{R [1] (t), . . . ,RW(t)}u°(t)+diag{EW(t), . . . , E^(t)}£°(t) 

+ diag{£ (1) '*0), ..., B {N) '*{t)}f°{t) + m{t) = 0. (54) 

Therefore, (1541 can be solved via fixed point methods. One can proceed further to determine the 
adjoint processes and the explicit optimal team strategy. This is done in the next subsection. 

A. Communication Channels with Memory and Feedback 

In this section we discuss applications of team games to communication channels with feed- 
back and memory. We consider applications in which the state process is a RV, and decentralized 
information structures with feedback and/or correlation among them. Consider a filtered proba- 
bility space (0,F, F T ,P) on which the following are defined. 



A Gaussian RV 9 = Vector{9\ ...,9 N }:Q — ► R n , 9 l e M n \ (E(0), Cov(9)) = (6, P ), 
Mutual Independent Brownian motions B l : [0, T] x04 M. ki , i = 1, ... TV, independent of 9. 

The information structure of each DM u % is Q^ t = a{y l (s) : < s < t},t E [0, T], i — 
1, . . . , N, which is defined by a communication channel with memory feedback via the stochastic 
differential equation 

y\t)= [ C ll (s,y\s))9ds+ [ DUs)dB\s), t e [0,T], i = 1, 2, . . . , N, (55) 

Jo JO 

where Cu : [0,T] x R hi — y C(M. n ,M. ki ). The communication channel (1551) models a Gaussian 
Broadcast channel in which there is a single transmitter and multiple receivers, i = 1, . . . , N. The 
transmitter wishes to send linear combinations of messages {9 1 , . . . , 9 N } to receivers {y 1 , . . . , y N }. 
The objective is to reconstruct at each receiver y % the intended linear combination of the 
messages denoted by L l 9, where L 1 is an appropriately chosen matrix. A reasonable pay- 
off for reconstructing the intended linear combination L l 9 at receiver i by u % is the average 
weighted estimation error E L „ (u t — diagjL 1 , . . . , L N }9, R(t)(u t — diagjL 1 , . . . , L N }9))dt. A 
more general pay-off which also incorporates any power constraints at the transmitter is the 
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quadratic pay-off defined by 



J{u\ ...,u N ) =^E{ J (<u t , R{t)u) + (9, H{t)9) + (9, F(t)} 

+ (u t ,E(t)0) + (ut,m(t)))dt. (56) 

Noticed that the information structures d55l are defined via channels with feedback since 



PJV(*) e Alibis) : < s < t - e},9~j ± p{^(£) G A|fl}, A e B(R ki ), e>0,teZ N . (57) 

The previous communication model can be easily generalized to other network communication 

channels. 

Since minimizing (l56l) over feedback information structures subject to (l55l) is a team problem, 

then we will apply the optimality conditions of Theorem |2] 

First, note that the stochastic differential equation (l55l has a continuous strong solution which 

is unique. Since the state is a RV (static state), then tp° = 0, hence the optimal strategies are 

given component wise by 



ur = -R^ 1 (t){j2E iJ (t)E(9>\gl)+m\t)+ Y, Mm(4'°\Q()}i P\^ t -a.8.,ieZ N . 

(58) 

Define the filter version of by 9\t) = e(#|($J) , £ G [0, T], i G Zjv. Then these bank of filters 
satisfy the following stochastic differential equations 

^(t) = P J (t,^Q(t,^(t))^ 1 (t)(^(t)-C rt (t,^(t))^(t)rft), 0*(O) = 0, te(0,T], ieZ N 

(59) 

P\t^) = -P\t^)C*^tJ{t))D^{t)C u {t,y\t))P%^\ P(0) = P , t G (0,21, i e Z*- 

(60) 
Define the innovations process and the er— algebra generated by it as follows. 

P(t) = J Dff\ S )(y\s)-C ii {s,y i {s))9\s)ds), te[0,T], ieZ N , (61) 

g^ t = a(p(s):0<s<t\, te[0,T\, ieZ N . (62) 

Then {P(t) : < t < T} is an (Q^, , Pj — adapted Brownian motion V? G Zyv, and for i 7^ j, 
the innovations P(-),P(-) are independent (in view of independence of B l (-), B^(-) for i ^ j). 
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Moreover, the processes {9\ P l (t,y l ),y l (t) : < t < T} are weak solutions [1381 of the system 

S\t) = P\t^)C* t (t,y\t))N^(t)dl\t), $(0) = 9, te(0,T], ieZ N (63) 

dy\t) = C ii (t,y i (t))9 i (t)dt + D$(t)dT(t), t e [0,T], i e Z N , (64) 

P\t,y l ) = -P%y i )C: i (t,y i (t))D^(t)C u (t,y i (t))P%y i ), P(0) = P , t G (0,T], i G Z N . 

(65) 
Next, we establish existence of strong solutions to the system (|63l)-(l63T) which will imply that 
G T and Gt* Vi e ^n generate the same information. Under assumption that Cu(t, y l ) satisfy the 
Lipschitz and linear growth conditions, uniformly in t G [0, T], the system (|63l) . (l64l) has a unique 
Gt~ adapted continuous solution [|38l . hence #*(•) is Gl\— measurable, Vi G [0, T], Vi G Zjv. 
Thus, <?£ C Go' t yt G [0,T], Vi G Z N . The reverse <?£ t C Gl] T M G [0,T], Vi G Zjy follows 
from the construction of innovations processes (16TT >. Hence, ^ = ^ , Vi G Zjv- Since each 
DM -u* is C/f, = Gt~ adapted Vi G Z^, and the innovations sigma algebras Gt are independent 
for Vi, j G Z/v, i ^ j then the optimal strategies (1581 ) are given by 



TV 

^ 1 (t)<j J B H (t)^(t) + m i (t)+ J^ %(t)E^f)}, P|^i -a.s.,ieZ N . (66) 

. ,E(U| '°)}. Taking expecta- 
tion of both sides of (l66l) gives the following linear system of equations. 



i,o 



Next, we determine the vector by u° = yector{E(n 1,0 ),E(M i '° 



N 



The last equation can be put into a fixed point form. Define 



i G Zm. 



(67) 



M(t) 



-R u \t)E^{t)E(e 
■R£(t)EW(t)E[0 



-R£ N (t)EW(t)E(e 



K(t) 



A 



■R^(t)m 2 (t) 



■R£r{t)m N {t) 



(68) 
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A(t) 



A 



^22 (t)R2l(t) 



R^(t)R 12 (t) R^(t)R 13 (t) 



R22 (t)R23(t) 



R u (t)RiN(t) 



R22 (t)R2N{t) 



RNN\t)RNl(t) RNNWRN2{t) R NN (t)R N3 (t) ... 

From (1671) . we have 



(69) 



A{t)v?{t) = M(t) + K{t), v°{t) =A- 1 (t)[M(t) + K(t)), te [0,T], ifA(t) >0,VtG [0,T].(70) 



Finally, the optimal strategies are given by (1661 ) and (TTOD . 

The previous calculations can be generalized to other channel models. Moreover, 9 can be 
extended to a Random process described by ltd stochastic differential equations. For linear 
dynamics this generalization is a straight forward repetition of the previous calculations, hence 
it is omitted. 



B. Linear-Quadratic Form and Linear Stochastic Differential Dynamics 

In this section we invoke the minimum principle to compute the optimal strategies, with respect 
to a quadratic pay-off, for distributed stochastic dynamical decision systems consisting of an 
interconnection of two subsystems, each governed by a linear stochastic differential equation 
with coupling. 
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Subsystem Dynamics 1: 

dx\t) =A 11 (t)x 1 (t)dt + Bni^uldt + Gu{t)dW l (t) 

+ A 12 (t)x 2 (t)dt + B 12 (t)u 2 dt, x l (0) = xl, te(0,T], (71) 

Subsystem Dynamics 2: 

dx 2 {t) =A 22 (t)x 2 (t)dt + B 22 (t)u 2 t dt + G 22 {t)dW 2 {t) 

+ A 2l (t)x l (t)dt + B 21 u]dt, x 2 (0) =x 2 , te (0,T] (72) 

For any £ G [0, T] the feedback information structure of u\ of subsystem 1 is the a— algebra 
@ot = cr {y 1 ( s ) : < s < t}, and the feedback information structure of u 2 of subsystem 2 is 
the a— algebra Q\ t = a{y 2 (s) : < s < t}. These information structures are defined by the 
following linear observation equations. 



Information structure of Local Control u 



i. 



y\t)= I C ll (s)x 1 (s)ds+ f D 1 \(s)d5 1 (s), t e [0,T]. (73) 

Jo Jo 



Information structure of Local Control u 2 



y 2 (t) = [ C 22 (s)x 2 (s)ds+ I Dl 2 (s)dB 2 (s), t e [0,T]. (74) 

Jo Jo 

We may also assume the DMs strategies u l and u 2 are functionals of the innovations information 
structures Q^f = a{/ 1 (s) : < s < t}, Q 1 ^ = cr{/ 1 (s) : < s < t] defined by the innovations 
processes of (l73l . (l74l . respectively. 



The pay-off or reward is quadratic in (x 1 , x 2 , u l , u 2 ). 
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J(u\u 2 ) 




,H(t) 



x\t) 
x 2 (t) 



) + ( 



+ (\ xl{T) U(T) ^ |». 

x 2 (T) I I x\T) 



ui 



u; 



R(t) 



ui 



u: 



dt 



(75) 



We assume that the initial condition x(0), the system Brownian motion {VF(t) : t E [0, T]}, and 
the observations Brownian motion {B l (t) : t G [0,T]}, and {B 2 (t) : £ e [0,T]} are mutually 
independent and x(0) is Gaussian (E(x(0)), Cou(a;(0))) = (xq,Po). 

For decentralized filtering we set u 1 = 0, u 2 = in the right hand sides of (|7TT ). (|72T ). but we 
should take as pay-off (l56l) . 



Define the augmented variables by 



x 



x 



j- 



a / y 



A / « 



U 



V> 



ir 



A 



w 2 



u 




5 



A 



B 1 

£ 2 



and matrices by 



.4 



A 



B 



G 



(i) * 



9n 






An A12 
A21 A 2 2 


, B^ 


-B11 -B12 
■B21 B22 


, c^ 







C22 


\ 


Bu 
B21 


5 


B< 2 >4 


B12 
B22 


, d 


L] A 


Cn 





, d 



A 



C- 



22 



A 



Gn 
G22 



D* 



l A 



£>f. 



22 



The distributed system is described in compact form by 

dx{t) =A{t)x{t)dt + B{t)u t dt + G(t)dW(t), x(0) = x t G [0,T], 
d y (t) =C(t)x(t)dt + L>^ (t)rfS(t) , t 6 [0, T] . 



(76) 

(77) 
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J(u\ u 2 ) = 1-e\ f (x(t),H{t)x(t)) + (ut, R{t)u t ) dt + (x{T),M(T)x{T)) \. (78) 



while the pay-off is expressed by 

1 

2 wo 
By Theorem |2l the Hamiltonian is given by 

H(t, x, if;, q n ,u) = (A(t)x + Bu, ^) + tr(q* n G(t)) + -(x, H{t)x) + ~{u, R{t)u). (79) 

The derivative of the Hamiltonian with respect to u component wise this is given by 

U u i(t, x, i/j, Q, u\ u 2 ) = B {1) '*(t)tP(t) + R xl {t)u x + R 12 (t)u 2 , (80) 

H„ 2 (t, x, xjj, Q, u\ u 2 ) = B^'*(t)ip(t) + R 22 (t)u 2 + R 21 {t)u\ (81) 

The optimal decision {u° t = (u t '°,u 2 -°) : < t < T} is obtained from (l8~0l) . (l8~TT ) by using the 
information structure available to each DM. 

Let (x°(-), ip°(-), ?ii(-)i Q° 2 (')) denote the solutions of the Hamiltonian system, corresponding 
to the optimal control u°, then 

dx°(t) =A(t)x°(t)dt + B(t)u° t dt + G(t)dW(t), x°(0) = x (82) 

dip°(t) = - A*(t)ip°(t)dt - H{t)x°{t)dt - V qli {t)dt 

+ q° u (t)dW(t) + q° 12 (t)dB(t), </>°(T) = M(T)x°(T) (83) 

Next, we identify the martingale term in (|83l . Let {$(£, s) : < s < t < T} denote the transition 
operator of A(-) and $*(•, •) that of the adjoint A*(-) of A(-). Then {^°(£) : t E [0,T]} is given 
by 

r(t)=<$>*(T,t)M(T)x°(T)+ J ^{s,t)\H{s)x°{s)ds + V q o 1 {s)ds 

~q° ll {s)dW{s)-q° 12 {s)dB{s)). (84) 

By using the using the identity J^$*(£, s) = — A*(s)§*(t, s), < s < t < T one can verify 
by differentiation that (l8~4l) is a solution of {ift°(-), q°i(-)i q° 2 (-)) governed by (l8~3T) . Since for any 
control policy, {x°(s) : < t < s < T} is uniquely determined from (|82l and its current value 
x°(t), then (l84l) can be expressed via 

V>°(t) = £(£)x°(£) + 0°(t), t e [0, T], (85) 
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where E(-),/3°(-) determine the operators to the one expressed via ([84}. 

Next, we determine the operators (E(-),/3°(-)). Applying the Ito differential rule to both sides 

of ([85]), and then using ([82]), ([83]) we obtain 

-A*(t)ifj°(t)dt - H(t)x°(t)dt - V qll {t)dt + q° n (t)dW(t) + q° 12 {t)dB(t) 

=t(t)x°(t)dt + E(t){A(t)x°(t)dt + B(t)u°dt + G(t)dW(t)\ + d(3°(t). (86) 

Substituting the claimed relation (1851 ) into (l86l) we obtained the identity 

- A*(i)E(£) - £(£)A(t) - H(t) - ±{t)\x°(t)dt - Vqo^dt + q° n (t)dW(t) + q° 12 (t)dB(t) 

=A*(t)/3°(t)dt + E(t)B(t)u° t dt + E(t)G(t)dW(t) + d/3°(t). (87) 

Since a(t,x) = G(t), then Vqo^t) = 0,Vt G [0,T]. By matching the intensity of the martingale 
terms {-}dW(t) in (1871) . and the rest of the terms we obtain the following equations. 

V^(£) = 0, Vte[0,T], (88) 

<&(*) = £(*)(?(*), £g[0,T], (89) 

E(£) + A*(£)E(£) + E(£)A(£)+/7(£) = 0, £G[0,T), E(T) = M(T), (90) 

d/3°(£) + A*(t)P°(t)dt + Y,{t)B{t)u° t dt - q° 12 {t)dB{t) = 0, £ G [0, T), f3°(T) = 0. (91) 

Notice that q° l2 is also obtained by Remark El since q? 2 (t) = ?p°(t)G(t) = E(t)G(t),Vt G [0,T]. 
By Theorem H {(Ui'°, w 2 '°) : < £ < T} obtained from ([80]) and (ED, are given by 



a.e.t G [0,71, p l y^° ~ a - s - ( 92 ) 

E{^ u2 (t, x 1 ^), * 2 '°(£), ^°(£), ^°(t), g!i°W, fti°W, tij' , ^'°)l<r° } = 0, 

a.e.t G [0,T], P| rV a,«» - o.s. (93) 

where (x°(-),f (-).9n(-)) = (^ 1 ' (-)^ 2 ' (-)^ 1 ' (-)^ 2 ' (-)^ii (-))^ii°(-)) are solutions of the 
Hamiltonian system (|82|) . (f83l) corresponding to u°. From (|92| ), (l93l the optimal decisions are 



^' 



1,0 



-i^WB^WEl^WISfi""} - i2T 1 1 (t)i2 u (t)E{ti?'°|SgJ'""} l t G [0,T]. (94) 
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2,0 



-i^WS^WEJ^WI^J-i^Wi^WE^^Sg'""}. te[0,T]. (95) 

Clearly, the previous equations illustrate the coupling between the two subsystems, since u l '° is 
estimating the optimal decision of the other subsystem u 2 '° as well as the adjoint processes ip° 
from its own observations, and vice-versa. 

Let </>(•) be any square integrable and F r — adapted matrix-valued process or scalar-valued 
processes, and define its filtered and predictor versions by 



A 



7r»(0)(t)^E|0(t)|0» t j, vr J (0)( S ,t) = E|0( S )|^), te[0,T], s>t, . = 1,2. 

For any admissible decision u and corresponding (x(-),ip(-) define their filter versions with 
respect to Q^ t for % — 1, 2, by 



A*)(t) 



E[x\t)\gi t 

E{x 2 (t)|C 



, 7T*(V)(*) 



A 



E{^(«, 



IT" 



t€[0,T], 2 = 1,2, 



T*(«)(*) 



A 



EWlC 



E «, 



i,t 



u 



1,i 



, te[0,T], i = l,2, 



and their predictor versions by 



7l\x)(s,t) 






(It 

o,/ 



, ^){s,t) 



A 



Ej^WieS 



E 



7r'(iO(M) 



A 



EKIfif 



ENW 






, te [0,T], s> t, i = l,2 



t G [0,T], s > £, z = l,2. 



From (1941) . (1951 ) the optimal decisions are 



1,0 



2,o 

Mi' 



-i^(t)B«'*(ty(^)(t) - Rn\t)Ri2(t)E{^VoT°}, t e [0,T], (96) 
■R£(t)B&>>*(t)**(iP)(t) ~ R^{t)R 21 {m{u\>°\g(;°), t e [0,T]. (97) 



The previous optimal decisions require the conditional estimates 

{(tt 1 ^ )^), 7i 2 (ip°)(t)) : < t < T}. These are obtained by taking conditional expectations of 

(f84l) giving 



T 



Ti\i)°)(t) = $*(T,t)M(T)ir l (x°)(T,t) + I <f>*(s,t)H( S y(x°)(s,t)ds, te[0,T], z = 1,2. 

(98) 
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For any admissible decision, the filtered versions of x(-) are given by the following stochastic 
differential equations ll38l . 

d7v\x){t) = A{t)'K 1 {x){t)dt + B {1 \t)u\dt + B^it^^ifydt + in\xx*){t) 

-7r 1 {x){t)Tr\x*){t)\c [1] '*D u 1 (dy\t)-C [1] {t)Tr 1 {x){t)dt), tt\x)(0) = x , (99) 



d-K 2 {x){t) = A{t)n 2 {x){t)dt + B {2 \t)u 2 t dt + B^i^iu^dt + U 2 (xx*)(t) 
- ii 2 (x)(t)Tt 2 (x*)(t))c [2] >*D^(t)(dy 2 (t) - C [2] (t)7r 2 (x)(t)dt), tt 2 (x)(0) = x . (100) 



From the previous filtered versions of x(-) it is clear that subsystem 1 estimates the actions of 
subsystem 2 based on its own observations, namely, 7r 1 (ti 2 )(-) and subsystem 2 estimates the 
actions of subsystem 1 based on its own observations, namely, ii 2 {u l )(-). 
For any admissible decision (u 1 ,^) G Vreg [0,T] define the innovation processes associated 
with {Gl t : t G [0,T]},i = 1,2 and the a— algebras generated by them as follows 

P(t) = fit) - ! C [l] (s)7i i (x)(s)ds, G^t = cr{P{s) : < s < t\, t G [0,7], i = 1,2(101) 

Let P'°(t) the innovations processes corresponding to where (x°,u°),i = 1,2. Then by (11011) , 

{P(t) : t G [0,T]} is ({Goj : t G [0,T]},PV adapted Brownian motion, P(t) has covariance 

Cov(P(t)) = f*D u (s)ds, for i = 1,2 and {P(t) : t G [0,T]}, {I 2 (t) : t G [0,T]} are 

independent. 

For any admissible decision u the predicted versions of x(-) are obtained from (l99l and (1100 1 ) 

as follows. Utilizing the identity 7r i (x)(s,t) = e|e|x(s)|^J]|^ u j = eJtt^xXs)^"}, 

for < £ < s < T then 

dTT 1 (x)(s, t) = A(s)n 1 (x)(s, t)ds + B {l \s)n l {u l )(s, t)ds + J B (2) (s)vr 1 (u 2 )(s, t)ds, t<s<T, 

(102) 

tt 1 ^)^,*) =7r 1 (s)(t), te[0,7), (103) 

rf7r 2 (x)(s, t) = A(s)tt 2 (x)(s, t)ds + j B (2) (s)vt 2 (m 1 )(s, £)ds + j B (1) (s)tt 2 (m 1 )(s, i)ds, t<s<7, 

(104) 

7r 2 (x)(£,f) =7 r 2 (2:)(t), £g[0,T). (105) 
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Since for a given admissible policy and observation paths, {7r 1 (a;)(s, t) : < t < s < T} is 
determined from (11021 ) and its current value 7r 1 (x°)(t,t) = 7r 1 (a;)(t), and {% 2 (x)(s,t) : < t < 
s < T} is determined from (11041 ) , and its current value ir 2 (x)(t,t) = ir 2 (x)(t), then (|98l) can 
be expressed via 

7r l (ij )(t) = iT(ty (x°)(t) + r l (t), t G [0,T], i = 1, 2. (106) 

where if*(-), r 4 (-) determines the operators to the one expressed via d98), for % = 1,2. Utilizing 
(TT061) into © and (J97]) then 

„!.• = -^(^W^tjJA^t), 1 !/)^) + r 1 ^)} - R u \t)R 12 (t)n\u 2 t >°)(t), t e [0,7], 

(107) 



2,o 



Uy- 



^ 1 (*)B«'*(t){l^(*y(^)(t) + r a (t)}-i^ 1 (t)i2 ai (t)7r , (« 1 ' )(t) ) tG[0,T]. 



(108) 

Let {^_K-i(t,s) : < s < t < T} denote the transition operator of A K i(t) = [A(t) - 
S^i)/^ 1 (£)£«'* (i)iT(t)Y for i = 1,2. 

Next, we determine K l (-),r l (-),i = 1,2. Substituting the previous equations into (11021 ). (11031 ) 
and (fT04l) . (flQ5T) then 

ir\x°)(s,t) =y K i(s,t)7r l (x°)(t)- I ^ K i(s,T)B^(r)R^ 1 (r)B^'*(T)r 1 (T)dT 

- J S ^ K i(s,T)B^(r)R u 1 (r)R 12 (r)n 1 (u 2 n(r,t)dT 

+ I ^> K i{s,T)B {2 \r)-K l {u 2 ^ ){T,t)dT, t<s<T, (109) 

ttVXM) =*MM)*V)(t) - j\ K ,{s,T)B^ 2 \T)R^{T)B^*{r)r 2 {r)dT 

- J\ K2 (s,T)B( 2 \T)R 2 7 2 1 (T)R 21 (T)7r 2 (u 1 <°)(T,t)dT 

+ f ^ K 2(s,T)B (1) (T)7l 2 (u 1 '°)(T,t)dT, t<s<T. (110) 

We now introduce the following assumption regarding the measurability of admissible decisions 

W i,a [0,T],i=l,2. 
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Assumptions 5. Any admissible decentralized feedback information structure u % E U yS ' u [0, T] is 
adapted to < Q 1 ^ : t E [0, T] >, i — 1, 2, and rt99l) faw a strong < Q^J" : £ e [0, T] >— adapted 
solution 7T 1 (x)(-) and diOOD /ms a strong < Q\^ : t E [0,T] >— adapted solution n 2 (x)(-). 



Now, we can state the first main result. 

Theorem 4. Under the conditions of As sumptions \5\ the optimal decisions (u 1,0 ,u 2, °) are given 
u]' = -i^)£ (1) '*(*){^W(*°)(*) + r\t)} - R£(t)Ru(t)u&(t), t E [0,T], (111) 



2,o 



Ut 



R^(t)B^*(t)\K 2 (t)ir 2 (x°)(t) + r 2 (t)\ - RgWRnQv^it), t E [0,T]. (112) 



w/?ere7r i (x°)(-),« = 1. 2 satisfy the filter equations rf99l). HWD. and I iT(-), r'(-), x°(-),« i,0 (-) ),* = 
1,2 are solutions of the ordinary differential equations A113\) . A114\) . A115\) . dll6\) . A117\) . A118\) . 

K\t) + A*(t)K l (t) + !T(t)A(t) - K i {t)B®{t)R£ 1 {t)B®>*(t)K i {t) 

+ H(t) = 0, tE [0,T), i = l,2, (113) 

K i (T) = M(T), z = l,2, (114) 

f!(t) ={ - A*(£) + $*(T,t)M(T)y K i(T,t)BW(t)R^(t)BM<*(t) 

^*(s,t)H(s)^ K x(s,t)ds)B ( - 1 \t)RY 1 l (t)B^'*(t)\r l (t) 

$'(s,t)ff(s)^(s,t)(is)(B( 2 )(t)-BW(t)^ 1 (f)i? u (t))^(t) 1 
- r(T,t)M(T)^i(T,t)(V 2 )(i) - B^(t)R^(t)R X2 (t))^(t) t E [0,T), r 1 ^) = 0, 






t 
T 



(115) 
f 2 (t) ={ - A*(t) + $*(T,t)M(T)q K 2(T,t)B {2) (t)R- 2 \t)B {2) '*(t) 

<S>%s,t)H(s)y K2 (s,t)ds}B( 2 \t)R^(t)BW>*(t)}r 2 (t) 

<S>*(s,t)H{s)y K 2{s,t)ds^ (#«(t) - B^{t)R 22 \t)R 21 {t)^u^(t) 

- $*(T,t)M(T)V K2 (T,t)(BW(t) - B^(t)R^ 2 l (t)R 21 (t))u^(t), t E [0,T), r 2 (T) = 0, 

(116) 
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x*{t) = Aitjx 5 ^) + 5 (1) (t)u 1 > °(t) + B {2 \t)u 2 >°(t), x*{0) = x , 



(117) 



u^°{t) 



u 2 >°(t) 



I R^{t)R l2 {t) 

R^ 2 \t)R 21 {t) I 



-1 r 



R£(t)B®>*(t)l K 2 (t)^(t) + r 2 (t) 



(.118) 



Proof: By invoking Assumptions [51 since y l (i) = J C^(s)ir(x)(s)ds + P(t), and n l (x)(-) 
is a strong solution then Q\' t C ^qT' ^ e [0>^1 an ^ thus £q 4 = £?ot" » * = 1>2. Hence, 
the optimality conditions of Theorem |2] are valid. Utilizing the independence of the innovations 
processes J 1 (-) and J 2 (-) then 



7r 1 (tt 2 )( S ,t) = E(u2|(?o"i)=E(«i 



u 2 (s), t<s<T, 



7t 2 (u 1 )(s,t) = Elu 1 s \g y ^) --E(ul) =vHs), t<s<T. 



(119) 
(120) 



Substituting (fTTPT) . ([1201) into (fl09l) . (fTTOl) . and then (1T091) . (ITTOl) into ([98j> we have 

7r 1 (r)(t)={^(T,t)M(T)^ K1 (T,t)+ f ^*(s,t)H(s)^ K1 (s,t)d s y i (x )(t) 

+ $*(T, t)M(T) | T ^(T, r) (i?( 2 )(r) - ^(r^V)^))^)^ 



+ f <S>*{s,t)H{s) f S ' V k i(8,t)(b®(t) - B^\r)R^{r)R l2 {r)y^{r)drds 
-$*(T,t)M(T) ( ^ K i(T,r)B^(r)R^(T)B^'*(ry(T)dr 

$*(s,t)H(s) f ^ K i(s,r)B^(T)R u 1 (r)B^'*(Ty(T)dTds, (121) 



T 

- J <J>*(s,t)H(s)V K 2(s,t)ds 



+ $*(T,t)M(T)j ^ K ,{T,r){B^{T)-B^{T)R^^{T)R 21 {r))u^{T)dr 
+ J <f>*(s,t)H(s) J ^ K 2(s,t)(b^(t) - B^(r)R 2 i(T)R 21 (rfju^(T)dTds 
- $*(T,t)M(T) J ^ K2 (T,r)B^(r)R^(r)B^*(r)r 2 (r)dr 



$*(s,t)H(s) / ^ K 2(s,r)B^(T)^(r)B^*(r)r 2 (T)dTds. 



(122) 
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Comparing (11061 ) with the previous two equations then K l (-),i = 1,2 are identified by the 
operators 

K i {t) = $*(T,t)M(T)V K i(T,t)+ f <S>*{s,t)H(s)^ K i{s,t)ds, t e [0,T], 2 = 1,2, 



(123) 
and r l (-),i = 1,2 by the processes 

rp 

r l (t)=$*(T,t)M(T)J 9jci{T,r)(B^{r)-B^\r)R^(T)Ru(r))^{T)dr 

+ I <!>*(s,t)H(s) I V k i(s,t)(b®(t) - B {l \r)R^(T)R 12 (T)y^(r)dTds 
-$*(T,t)Af(T) I ^ K i(T,T)B^(T)R n 1 (T)B^'*(T)r 1 (r)dT 
■ ( $*(s,t)H{s) ( ^ K i(s,T)B^(T)R^ 1 1 (T)B^*(T)r l (T)dTds, (124) 

r\t) =<l>*(T,t)M(T) j T ^ K2 (T,r)^\r) - B^\r)R^(r)R 21 (r)^ {r)dr 

+ I $*(s,t)H(s) I ^ K 2(s,t)(b (1 \t) - B^(T)R- 2 1 (r)R 21 (T)^u^(T)dTds 
-&(T,t)M(T)J ^ K ,{T,r)B^\r)R^{T)B^\r)r 2 {r)dT 

- f $*(s,t)H(s) ( ^ K 2(s,T)B^(T)R^\T)B^*(T)r 2 (r)dTds. (125) 

Differentiating both sides of (11231 ) the operators K l (-),i = 1,2 satisfy the following matrix 
differential equations (111 3b . (11141) . Differentiating both sides of (11241) . (11251) the processes 
r*(-),i = 1,2 satisfy the differential equations (11151) . (11161) . Utilizing ( 11191) . (11201) we obtain 
the optimal strategies (11111) . (11121) . Next, we determine u^° for i— 1,2 from (|1 1 1|> . (|112|) . 
Define the averages 

x(t) =e{ie(£)} =EU i (x)(t)\, i = l,2. (126) 

Then x°(-) satisfies the ordinary differential equation (11171 . Taking the expectation of both sides 
of (11111) . (11121) we deduce the corresponding equations 

u^(t) = -R^^B^^tK^t^it) + r x (£)} - R^(t)R 12 (t)u^(t), t e [0,T], (127) 

7&(t) = -R 22 \t)B^*{t){K 2 {t)-°{t) + r 2 (£)} - R£{t)R 21 {t)u&{t), t e [0,T]. (128) 



it- 
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The last two equations can be written in matrix form (11181 . This completes the derivation. ■ 
Hence, the optimal strategies are computed from (11111 ). (11121) . where the filter equations for 
7 r i (x°)(-),i = 1,2 satisfy d99j), (flOOl) . while (K\-),r\-)^(-)^(-)\i = 1,2 are computed 
off-line utilizing the ordinary differential equations (11131) . (11141) . (11151) . (11161) . (11171) . (11181) . 

It is important to make the following observations. 

(Ol): The optimal strategies or laws (11111 ). (11121 ) are precisely the optimal strategies obtained 
in ll39l for noiseless decentralized information structures. This property is analogous to that of 
optimal centralized strategies of fully and partially observed Linear-Quadratic-Gaussian systems. 

(02): The filter equations for ir l (x°)(-),i = 1,2 given by (l99l) . (11001) are nonlinear and may 
require higher order moments, leading to the so-called moment closure problem of nonlinear 
filtering. Furher analysis is required to determine whether the conditional error covariance in 
(1991) . (11001) have the Kalman filter form. 

Next, we state analogous results for distributed filtering problems. 

Corollary 2. Consider distributed filter dynamics ([77]), ( f72l with B^ = 0,i = 1,2 and LQF 

pay-off m 

Then the optimal strategies (u 1 ' , u 2, °) are given by 

2 2 

«j'° = - R^(t){m*(t) +J2 E M®(x J (t)\Gi) + E Rdm(<)}, % f - o.s., Vz e Z 2) 

(129) 



where x*(t) = Vector < E{x 1 (t)|^Q t },E{x 2 (t)|^Q < } L z = 1,2 satisfy the linear Kalman filter 
equations 

dx\t) =A(t)x l (t)dt + P^CM'*!^) (V(t) - C [l \t)x l {t)dt\ x(0) = x , (130) 

d£ 2 (£) =A(£)£ 2 (£)dt + P 2 {t)C^'*D^{t)(dy 2 {t) - C [2] (t)x 2 (t)dty £°(0) = i , (131) 

P\t) =A(t)P i (t) + P l (£)A(£) - P 2 (£)C ,H '*(£)P^ 1 (£)C' [il (£)P 2 (£) 

+ G(*)G*(t), P l (0) = P \ i = 1,2. (132) 

andu(t) = Vectorl E{tiJ},E{w 2 } > satisfy the equations 



d 

R(t)u°(t) + E(t)x(t) + m(t) = 0, -f~x(t) = Ax(t), x(0) = x . (133) 
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Proof: (11291 ) is obtained from (1531 by setting £?W — o, iV = 2, and the discussion in Sec- 
tion IV-AI (for filtering problems the observations and innovations generate the same filtrations). 
The filters (11 301 >- <11 32b are follow from the linear and Gaussian nature of the state and observation 
equations. Taking expectation of both sides of (11291) yields (11 33b . ■ 

Finally, we state a remark describing extensions of the previous examples. 

Remark 4. Theorem |4] is easily generalized to the following arbitrary coupled dynamics 

dx\t) =A ii (t)x i (t)dt + B^uldt + GadW^t) 

N N 

+ J2 AjX j (t)dt+ J2 B {j) (t)u{dt, x \o) = x , te(o,T], ieZ N (134) 

and information structures generated by observation equations with feedback 

y l = [ C ii (s,y i (s))x(s)ds + [ Dl{s)dB\s), te[0,T], ieZ N . (135) 

Jo Jo 

The optimal strategies are extensions of the ones given in Theorem H] Similarly, one can 
generalize the filtering results of Corollary \2\ to the above models with B^ % ' = 0. 

VI. Conclusions and Future Work 

In this paper we have considered team games for distributed stochastic differential decision 
systems, with decentralized noisy information patters for each DM, and we derived necessary and 
sufficient optimality conditions with respect to team optimality and person-by-person optimality 
criteria, based on Stochastic Pontryagin's minimum principle. 

However, several additional issues remain to be investigated. Below, we provide a short list. 

(Fl) In the derivation of optimality conditions we can relax some of the assumptions by 
considering spike or needle variations instead of strong variations of the decision 
strategies (or use relaxed strategies as in [0). Moreover, for team games with non- 
convex action spaces A\i = 1,2, ... ,N and diffusion coefficients which depend on 
the decision variables it is necessary to derive optimality conditions based on second- 
order variations. 

(F2) The derivation of optimality conditions can be used in other type of games such as 
Nash-equilibrium games with decentralized noisy information structures for each DM, 
and minimax games. 
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(F3) It will be interesting determine whether (1991 ) and (11001 ) are given by the Kalman filter 
equations. 
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